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EXECUTIVE  SUMMARY 


A  test  and  evaluation  (effort  or  study)  was  conducted  under  the  Tech  Solution  program  of  the 
Office  of  Naval  Research  to  determine  the  feasibility  and  effectiveness  of  wireless  wearable 
computing  systems  for  support  of  aviation  maintenance  technicians.  The  body-worn  information 
systems  included  electronic  technical  manual  support,  a  virtual  (i.e.,  software-emulated)  test 
equipment  suite,  wireless  voice  communications,  and  Naval  Aviation  Logistics  Command 
Management  (NALCOMIS)  database  connectivity,  coordinated  over  a  wireless  local-area  network 
(LAN). 

The  program  was  conducted  at  Naval  Air  Station  Brunswick  between  August  2002  and  December 
2003,  and  involved  squadrons  VP-10  and  VP-26.  Testing  included  objective  (e.g.,  completion  time, 
troubleshooting  accuracy,  etc.)  and  subjective  (user  evaluation)  methods  that  compared  performance 
between  conventional  tools  and  wearable  computers. 

Obtaining  authorization  to  operate  the  wireless  network  in  an  operational  facility  was  the  most 
troublesome  program  issue,  and  delayed  data  collection  considerably.  Although  the  network  was 
successfully  installed,  the  delays  exhausted  project  funding  just  as  data  collection  began,  although 
only  limited  user  feedback  was  gathered,  this  information,  was  sufficient  to  make  the  following 
observations  about  wireless  system  use: 

•  Electronic  maintenance  manuals  are  useful,  although  no  new  insights  were  gained  through  their 
integration  with  wireless  systems. 

•  Virtual  test  equipment  and  remote  interaction  with  NALCOMIS  are  the  two  capabilities  with  the 
greatest  potential  for  maintenance  support  over  wireless  networks. 

•  Wireless  support  may  not  require  wearable  computers;  alternate  information  systems  may  be  just 
as  effective. 
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BACKGROUND 


The  main  purpose  of  Tech  Solution  00039  was  to  demonstrate  and  evaluate  an  information  system 
to  support  aviation  maintenance.  The  proposed  technical  solution  focused  around  a  set  of  diagnostic, 
communication,  and  decision-support  tools  integrated  into  a  portable  computer,  with  collaboration 
support  furnished  through  a  wireless  network.  SPAWAR  Systems  Center,  San  Diego  (SSC 
San  Diego),  in  partnership  with  Bath  Iron  Works  (BIW),  initiated  a  project  to  satisfy  the  needs  of  the 
Office  of  Naval  Research  (ONR)  Tech  Solution  through  a  program  of  software  integration,  structured 
field  evaluation,  and  system  definition  to  establish  a  foundation  for  service-level  funding  (i.e., 
program  transition)  of  a  distributed  wireless  wearable  computing  (WWC)  information  system. 

Specifically,  the  project  sought  to  examine  knowledge  extension  (KE)  through  voice,  video,  and 
data  exchange  among  aviation  maintenance  personnel  using  WWC  systems  hosting  the  following: 

•  Interactive  electronic  technical  manuals  (IETMs)  that  provide  maintenance  technicians  with 
diagnostic,  troubleshooting,  and  procedural  task  guidance  at  the  work  site.  While  IETMs  are 
already  deployed  in  units  of  the  U.S.  Navy  and  Marine  Corps,  their  use  on  a  personal,  body-worn 
computer  was  new. 

•  Wireless  communications  tools  (i.e.,  Voice-over-Internet  Protocol  [VoIP])  that  permit  real-time 
consultation  with  other  technicians  and  hangar  work  centers  without  leaving  the  work  site. 

•  Virtual  test  equipment  (VTE)  (i.e.,  measurement  and  diagnostic  systems  integrated  into  the 
WWC  platform).  The  goal  was  to  reduce  the  amount  of  support  equipment  that  technicians  were 
required  to  carry  to  the  work  site. 

An  additional  objective  cited  in  the  original  Tech  Solution  solicitation  was  to  expedite  fleet 
introduction  of,  and  identify  a  funding  path  for,  WWC  and  VTE  technologies.  Achieving  this 
objective  involved  establishment  of  a  technical  context  and  business  case  for  these  technologies 
through  comprehensive  data  collection  and  analysis. 

Project  issues  surrounding  design  of  the  Tech  Solution  fell  into  two  classes:  programmatic  and 
scientific. 

PROGRAM  ISSUES 

System  integration  was  addressed  from  the  beginning  of  project  design.  While  IETMs  have  a 
longer  development  history  than  WWC  and  VTE,  electronic  manuals  can  be  even  more  effective 
when  combined  with  these  newer  technologies.  The  need  for  the  Tech  Solution  project  was  to  ensure 
that  WWC,  VTE,  and  IETMs  could  be  organizationally  integrated  into  a  coherent  system  of  distrib¬ 
uted  information  support. 

The  engineering  process  of  implementing  a  wireless  network,  and  quantifying  its  performance 
(e.g.,  establishing  the  reliability,  signal  coverage,  and  data  loading  characteristics)  in  an  operational 
environment  could  guide  the  efficient  deployment  of  future  networks  and  lead  to  the  realization  of  all 
the  technologies  that  could  ride  on  these  networks.  Wireless  network  security  and  integrity  were 
appreciated  even  before  work  began.  Information  Technology  (IT)  managers  throughout  the  Depart¬ 
ment  of  Defense  (DoD)  are  extremely,  and  justifiably,  sensitive  to  the  need  for  data  security  over 
wireless  or  radio  frequency  (RF)  communications  networks.  Although  the  Tech  Solution  project 
planned  to  use  virtual  private  networks  (VPNs)  to  provide  needed  security,  the  certification  process 
that  permits  network  operation  was  long  and  complex.  The  certification  process  formed  a  major 
component  of  the  project  design;  team  members  understood  that  this  process  would  limit  develop¬ 
ment  of  the  infrastructure  on  which  the  entire  project  depended. 
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Introduction  of  any  new  technology  carries  two  implicit  objectives:  (1)  to  ensure  that  current  users 
see  its  value  (i.e.,  that  users  “buy  in”  to  the  technology),  and  (2)  to  ensure  that  a  foundation  for  the 
future  is  laid  through  a  mutual  exchange  of  ideas  between  users  and  developers.  Therefore,  while  the 
focus  of  the  Tech  Solution  is  to  validate  the  utility  of  the  WWC/VTE  suite,  the  Tech  Solution  Team 
sought  to  exploit  this  opportunity  to  identify,  structure,  and  prioritize  issues  for  future  technology 
development  efforts. 

SCIENCE  ISSUES 

The  enterprise-level  impact  of  WWC  and  VTE  technologies  is  not  well  defined  because  little 
xperience  has  been  gained  with  either  class  of  system.  The  science  domain  is  wide  open  to  new 
models  of  maintenance  that  can  drive  the  effectiveness  of  these  technologies  when  they  are 
eventually  deployed.  For  this  reason,  such  issues  represent  a  great  transformational  opportunity. 
While  the  current  Tech  Solution  can  only  highlight  the  implications  of  these  topics,  and  perhaps 
guide  future  work,  such  treatment  fills  many  gaps  that  are  not  addressed  in  other  maintenance 
support  programs. 

SSC  San  Diego  approached  the  design  of  the  Tech  Solution  project  with  a  solid  base  of  experience. 
As  the  primary  human  interface  development  laboratory  for  the  Department  of  the  Navy  (DoN),  SSC 
San  Diego  possessed  extensive  experience  in  design,  execution,  and  analysis  of  user  tests  for  new 
technologies.  Specifically,  SSC  San  Diego  was  involved  in  WWC  for  over  4  years,  and  has  success¬ 
fully  executed  funded  projects  involving  the  following: 

•  Hand-held,  body-worn,  and  head-mounted  (augmented  reality)  displays  for  mobile  personnel 
support  in  indoor  and  outdoor  environments 

•  Shipboard,  shore-based,  and  field  variants  of  robust  wireless  data  networks,  including 
electromagnetic  interference  (EMI)  and  security  certification  requirements 

•  Hands-free  system  interfaces  incorporating  voice  and  gesture  control 

•  Electronic  distance  support  for  maintenance  activities;  SSC  San  Diego  is  currently  involved  in 
the  Distance  Support/Telemaintenance  Advanced  Concept  Technology  Demonstration  (ACTD) 
program 

•  Electronic  maintenance  databases  and  information  infrastructures;  SSC  San  Diego  manages 
several  configuration  and  update  projects  for  U.S.  Navy  IETM  programs 

The  desired  end-state  of  this  Tech  Solution  was  a  clear  implementation  model  backed  up  by  a 
systematic,  credible  record  of  testing  success. 
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METHOD 


The  Tech  Solution  implemented  a  fully  functioning  maintenance  support  system  consisting  of  a  set 
of  wireless  wearable  computers — each  containing  an  electronic  maintenance  manual,  VoIP,  and  VTE 
subcomponents — integrated  with  a  secure  wireless  network  and  usable  throughout  the  entire  range  of 
squadron  spaces  (i.e.,  hangar,  work  centers,  and  flight  line).  Three  perfomiance  topics  were  identified 
for  study  in  order  to  develop  a  comprehensive  characterization  of  the  system — performance  effec¬ 
tiveness,  user  evaluation,  and  engineering  factors.  The  Tech  Solution  project  used  quantitative  and 
qualitative  measures.  A  self-reporting  (user  survey)  method  was  used  so  that  large  data  sets  could  be 
economically  gathered  during  the  study. 

As  described  in  the  original  Tech  Solution  proposal,  performance  metrics  addressed  hardware, 
software,  and  usability — with  a  clear  emphasis  on  usability.  Measurement  topics  were  developed  to 
include  the  following: 

•  Ease  of  equipment  use  (i.e.,  can  the  equipment  be  accessed  and  controlled  alongside  existing 
maintenance  equipment  and  in  the  expected  settings  such  as  cockpits,  flight  lines,  and 
workbenches?) 

•  Training  time  for  operators  and  data  analysts  (e.g.,  how  quickly  can  users  become  proficient? 

Can  users  retain  that  proficiency  between  shifts  or  tasks?) 

•  Efficiency  and  ease  of  access  to  support  information  (i.e.,  are  the  users  getting  the  assistance  they 
need  from  the  WWC  and  its  underlying  information?) 

•  Ease  of  interaction  under  varying  environmental  conditions  (e.g.,  indoor-outdoor,  day-night, 
etc.) 

•  Effectiveness  as  a  function  of  maintenance  task  (e.g.,  system  specialty,  task  complexity,  etc.) 

•  Reach-back  effectiveness  (i.e.,  work  center,  database,  and  team  member  interaction  effects) 

•  User  acceptance  and  suggestions  for  changes/improvements  (we  were  looking  for  problem 
solutions  as  well  as  ideas  for  new  applications  or  features) 

•  Quantitative  measures  of  time  and  accuracy  (used  for  analyses  of  potential  cost  savings) 

PERFORMANCE  EFFECTIVENESS 

The  fundamental  contribution  that  the  Tech  Solution  can  provide  to  the  U.S.  Navy  is  quantitative 
metrics  regarding  the  performance  effectiveness  of  the  system  (and/or  its  components).  These  metrics 
involve: 

•  Time  to  complete  a  task 

•  Frequency  of  face-to-face  (versus  on-site)  interactions  with  supervisors  or  work  centers  to 
complete  problem  analyses 

•  Time  spent  obtaining  tools  to  complete  a  task  after  the  problem  has  been  diagnosed 

Reductions  in  any  or  all  of  these  metrics  would  demonstrate  the  potential  utility  of  the  WWC 
solution.  The  Tech  Solution  plan  called  for  generation  of  a  baseline  data  set  for  these  metrics 
collected  from  technicians  using  conventional  methods  and  tools,  compared  with  a  later,  matched 
data  set  collected  from  technicians  using  WWC  support. 
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USER  EVALUATION 

Some  characteristics  of  technology — and  the  impact  they  have  on  work  practices — cannot  be 
identified  with  observable  performance  metrics;  user  opinion  is  then  used  as  a  method  for  evaluating 
the  “intangibles.”  While  the  U.S.  Navy  requires  clear  performance  benefits  from  a  new  technology  to 
justify  acquisition,  user  acceptance  of  the  technology  is  still  necessary  or  it  will  not  be  used. 
Therefore,  the  Tech  Solution  project  includes  a  user  evaluation  survey  to  solicit  judgment  data  from 
technicians  employing  WWC.  To  capture  any  changes  in  opinion  owing  to  increasing  familiarity, 
this  survey  was  designed  for  use  at  the  beginning  and  end  of  the  study. 

ENGINEERING  FACTORS 

The  process  required  to  instantiate  a  functional,  secure  wireless  network  at  an  operational  military 
facility  is  not  clearly  defined,  owing  to  the  fluctuating  guidance  emerging  throughout  DoD  IT 
agencies.  This  climate,  in  turn,  is  complicated  by  the  heightened  concern  for  data  security  following 
9/1 1,  by  the  poorly  understood  effects  of  operational  wireless  communications  transmissions  and  the 
abundance  of  devices  (e.g.,  cell  phones,  personal  digital  assistants  [PDAs],  etc.)  that  use  them,  and 
the  emerging  procedural  complications  associated  with  the  Navy-Marine  Corps  Intranet  (NMCI). 

The  known  steps  required  to  establish  an  approved  network  at  the  test  site  (Naval  Air  Station  [NAS] 
Brunswick)  included: 

•  Completing  a  physical,  electrical,  and  environmental  survey  of  the  hangar  and  squadron  spaces 
used  for  the  Tech  Solution  test 

•  Developing  and  documenting  a  security  policy  (a  System  Security  Authorization  Agreement 
[SSAA])  and  obtaining  approval  of  the  plan 

•  Obtaining  written  command  approval  for  network  operation  from  the  IT  offices  of  CPRW-5, 
NAS  Brunswick,  and  Navy  Region  North  East  (CNRNE)  to  include  Interim  Authority  to  Operate 
(IATO)  certification 

•  Obtaining  system  safety  certifications  (e.g.,  EMI  and  Hazards  of  Electromagnetic  Radiation  to 
Ordnance  [HERO]  inspection  documents) 

•  Developing  a  plan  for  storing,  accounting  for,  and  maintaining  WWC  units  in  the  hangar  spaces 

Because  an  effort  like  this  had  not  been  attempted  before,  the  design  and  authorization  process 
developed  as  the  project  evolved.  The  data  for  this  aspect  of  the  work  were  therefore  descriptive,  and 
the  result  of  this  effort  was  a  procedural  “go  by”  with  appropriate  “lessons  learned”  for  future 
network  installations. 

The  original  Tech  Solution  proposal  specified  a  review  of  multiple  systems  (e.g.,  PDAs,  laptops) 
to  establish  a  scale  for  evaluating  WWC  performance  against  alternative  technologies.  This  topic  was 
addressed  through  selective  questions  in  the  User  Evaluation  surveys  (described  below)\to  determine 
which  performance  gains  (if  any)  were  realized  through  the  overall  network  solution  and  which  were 
unique  to  the  WWC  systems.  In  particular,  items  such  as  3  and  4  of  the  WWC/VTE  System 
Evaluation  (Appendix  A- 1 )  were  included  in  the  survey  instruments  to  solicit  user  reaction  to  these 
issues.  In  addition,  hardware  and  software  performance  characteristics  (e.g.,  battery  life,  weight, 
application  reliability,  etc.)  are  necessary  and  critical  metrics  to  any  evaluation  of  a  new  technology. 
User  opinion  on  these  factors  was  solicited  through  such  items  as  1,2,  9,  and  10  in  the  survey. 
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PROTOCOL 

The  data  collection  procedure  was  executed  according  to  the  following  protocol.  Figure  1  shows 
the  WWC  system  configuration  used  for  this  project. 


Figure  1 .  WWC  configuration  for  maintenance  support. 


Initial  Interviews 

The  WWC/VTE  System  Evaluation  (Appendix  A-l)  was  designed  to  gather  user  reaction 
information  regarding  the  WWC  and  its  utility.  This  report  was  first  administered  to  prospective 
WWC  users  as  soon  as  their  training  was  completed  to  establish  an  opinion  baseline.  An  interviewer 
from  SSC  San  Diego  or  Bath  Iron  Works  (BIW)  went  over  the  form  with  each  user  to  make  sure  that 
users  understood  the  questions.  These  reports  took  approximately  20  minutes  to  complete. 

Baseline  Performance  Characterization 

Maintenance  personnel  were  instructed  to  complete  a  Maintenance  Task  Results  report  (Appendix 
A-2)  at  the  conclusion  of  every  maintenance  action.  Personnel  were  trained  in  how  to  use  this  form 
before  first  use,  and  the  data  generated  over  a  5 -week  period  were  gathered  to  establish  a  perform¬ 
ance  baseline  of  maintenance  actions  using  conventional  methods.  Participants  could  complete  the 
form  in  about  3  to  4  minutes,  with  minimal  interference  with  operational  job  duties.  These  forms 
were  gathered  and  maintained  by  Squadron  Maintenance  Control  for  weekly  pickup  by  BIW 
personnel. 

On-the-Job  WWC  Use 

Once  maintenance  personnel  had  been  trained  in  WWC  use  and  had  completed  an  initial 
WWC/VTE  System  Evaluation,  users  were  trained  in  how  to  use  the  WWC/VTE  Results  report 
(Appendix  A-3).  Participants  could  also  complete  this  form  in  less  than  4  minutes.  Once  WWC 
systems  were  in  use,  technicians  were  asked  to  complete  a  Maintenance  Task  Results  report  and  a 
WWC/VTE  Results  report  for  every  job  where  WWC  was  used.  The  WWC/VTE  Results  report 
included  survey  items  unique  to  the  performance  of  the  technology  (e.g.,  its  availability,  utility, 
specific  strengths  and  weaknesses  in  the  context  of  the  maintenance  task,  etc.).  These  forms  were 
gathered  and  maintained  by  Squadron  Maintenance  Control  for  weekly  pickup  by  BIW  personnel. 
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Late-Term  Interviews 

The  WWC/VTE  System  Evaluation  was  administered  for  each  technician  approximately  8  weeks 
after  WWC  use  began  (for  that  individual). 

Post-Session  Interviews 

After  all  data  were  collected,  a  final  exit  interview  was  administered  to  identify  issues,  concerns, 
and/or  suggestions  by  individuals  that  did  not  find  their  way  onto  the  survey  instruments.  All  reports 
and  interviews  were  examined  for  trends  as  a  function  of  on-the-job  experience  with  the  system. 

Because  BIW  and  its  partners  provided  the  technologies  for  this  Tech  Solution,  SSC 
San  Diego  acted  as  the  impartial  evaluator  for  data  collection  and  analysis.  SSC  San  Diego  did, 
however,  assist  with  user  orientation  and  training  to  ensure  that  the  data  collection  effort  got  off  to  a 
sound  start,  and  supplemented  the  study  with  on-site  visits  at  selected  intervals  in  the  project 
preparation  and  data  collection  process. 
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RESULTS 


The  Tech  Solution  project  began  in  August  2002  and  ended  in  September  2003. 

OVERVIEW 

The  project  was  initiated  in  August  2002  after  a  contract  award  to  BIW,  and  baseline  data 
collection  began  the  same  month.  Engineering  installation  of  the  wireless  network  commenced  in 
October  2002  in  parallel  with  certification  efforts.  Interagency  discussions  and  policy  setting  delayed 
submission  of  certification  documents  until  Winter  2003.  Authority  to  operate  was  not  received  until 
early  summer.  WWC  training  began  immediately,  and  data  collection  commenced  in  July.  Funding 
resources  were  depleted  in  September,  but  data  collection  continued  until  the  project  concluded  in 
November.  The  following  sections  summarize  the  major  milestones  of  the  Tech  Solution  project. 


TIMELINE 

March 

2002 

SSC  San  Diego  Tech  Solution  proposal  submission 

April 

2002 

BIW  response  to  Tech  Solution  request  for  proposal  (RFP) 

August 

2002 

BIW  Tech  Solution  contract  award 

August 

2002 

Baseline  Job  Report  data  collection  from  VP- 10 

October 

2002 

BIW  Site  Survey  Report 

October 

2002 

BIW  System  Design  Document 

December 

2002 

HERO  certification 

March 

2003 

SSAA  submission  to  CPRW-5 

April 

2003 

IATO  document  submission 

May 

2003 

EMI  effects  approval 

May 

2003 

IATO  endorsement  by  Naval  Network  Warfare  Command 
(NETWARCOM) 

June 

2003 

IATO  approval  by  CPRW-5  (6  months  of  operation) 

July 

2003 

WWC  General  System  Form  initial  data  collection  at  VP-26 

September 

2003 

Project  funding  depleted 

November 

2003 

WWC  General  System  Form  final  data  collection 

December 

2003 

Final  data  set  delivery  to  SSC  San  Diego 

Difficulties  in  obtaining  permission  to  operate  the  wireless  network  drove  the  project  schedule. 

The  Tech  Solution  was  executed  during  administrative  confusion  regarding  DoD  security  policy, 
which  was  undergoing  discussion  and  debate,  with  multiple  agencies  taking  precautionary  positions 
of  delay  while  service-wide  guidance  was  generated.  In  addition,  NMCI  transition  was  initiated  at 
NAS  Brunswick  during  this  time,  and  uncertainty  regarding  its  implications  exacerbated  the  approval 
process.  The  practical  affect  of  these  problems  was  that  each  level  of  the  approval  chain  hesitated  to 
commit  to  an  authorization  strategy,  reluctantly  exercised  an  authorization  strategy,  or  overtly 
resisted  the  project  entirely.  Consequently,  considerable  effort  was  expended  in  working  the 
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authorization  request  through  the  command  chain.  Permission  to  operate  the  network  was  not 
obtained  until  June  2003. 

As  shown  in  the  timeline,  WWC  data  collection  efforts  began  as  soon  as  authorization  was 
received.  Although  the  startup  process  encountered  the  kinds  of  technical  difficulties  typical  of 
research  efforts,  and  some  weeks  were  spent  in  debugging  the  hardware  and  the  software,  data 
collection  was  conducted  when  the  system  was  available.  Given  the  protracted  interval  involved  in 
the  authorization  process,  however,  and  the  extensive  manpower  commitments  required  to  define  and 
complete  it,  the  project  became  funding-limited.  Support  for  the  project  ended  in  September  2003. 
The  Tech  Solution  effort  concluded  its  final  data  collection  efforts.  ONR  instructed  the  project  to 
assign  responsibility  for  the  WWC  equipment  to  the  squadron. 

PERFORMANCE  EFFECTIVENESS 

The  Tech  Solution  protocol  measured  WWC  system  performance  effectiveness  by  comparing  two 
groups:  technicians  using  conventional  maintenance  methods  (members  of  Squadron  VP- 10)  and 
technicians  using  WWC  and  VTE  support.  As  Appendix  A-2  shows,  the  following  performance 
metrics  were  included: 

•  Time  to  complete  a  task,  categorized  by 

Type  of  task 

Location  (e.g.,  hangar,  flight  line) 

Time  of  day 

Technical  rate(s)  performing  the  job 

•  Tools  required  (to  assess  the  utility  and  desired  future  capabilities  of  VTE) 

•  Transit  required  (i.e.,  consultation  trips  back  to  the  work  center  or  Maintenance  Control  (which 
could  be  reduced  through  WWC  communications) 

The  baseline  data  set,  representing  conventional  maintenance  methods,  was  collected  from 
members  of  VP- 10.  Over  a  5-week  period,  241  Maintenance  Task  Results  reports  were  gathered 
from  eight  technical  rates. 

As  mentioned  earlier,  the  WWC  break-in  period  at  VP-26,  July-September  2003,  encountered 
technical  difficulties  that  made  WWC  availability  intermittent.  Furthermore,  the  command  insisted, 
and  the  Tech  Solution  Team  agreed,  that  a  wireless  connection  to  NALCOMIS  (the  electronic 
maintenance  action  recording  system)  was  necessary  to  fully  exploit  WWC  applications.  Integration 
and  validation  of  this  feature  took  additional  time  away  from  WWC  availability.  The  team  declined 
to  collect  Maintenance  Task  Results  data  during  this  interval,  choosing  instead  to  wait  until  WWC 
systems  could  be  used  reliably  on  a  regular  basis  (where  the  data  would  be  more  accurate  and  more 
meaningful).  Unfortunately,  project  funding  was  depleted  before  this  data  collection  could  begin.  As 
a  result,  no  comparison  data  were  available  to  complete  this  portion  of  the  project  analysisAS'C  San 
Diego  retained  baseline  data,  however,  pending  any  future  effort  to  measure  WWC  on-the-job 
performance. 

USER  EVALUATION 

In  July  2003,  the  Tech  Solution  Team  collected  WWC/VTE  System  Evaluations  from  16  Aviation 
Electronics  (AT)  technicians,  the  first  group  of  WWC-trained  personnel  at  VP-26.  These  data 
represent  user  judgments  about  WWC  and  VTE  after  system  orientation,  but  before  on-the-job  use. 
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The  objective  of  this  survey  was  to  gather  “first-look”  evaluations  that  could  then  be  compared  with 
identical  surveys  collected  6  to  8  weeks  later,  after  extended  use. 

Again,  technical  difficulties  in  implementing  regular  WWC  functionality  and  delays  to 
accommodate  NALCOMIS  functionality  interfered  with  the  development  of  a  solid  base  of  user 
experience.  Final  administration  of  the  WWC/VTE  System  Evaluation  (from  14  technicians)  was  not 
completed  until  the  end  of  October  2003,  an  interval  of  approximately  16  weeks.  The  results  of  both 
surveys,  including  comments  from  the  personnel  who  completed  the  forms  are  in  Appendix  A-l. 

Interpreting  responses  from  such  a  small  group  is  difficult;  individuals  with  strong  opinions  can 
influence  the  final  scores,  when  averaged.  Nevertheless,  the  following  useful  observations  could  be 
made  from  these  data: 

•  The  WWC  system  is  light,  but  possibly  too  bulky  (note  that  design  changes  since  this  Tech 
Solution  was  conducted  may  have  already  improved  this  condition). 

•  System  reliability  made  an  impression  on  the  users;  the  technical  difficulties  in  standing  up  the 
full  WWC  capability,  including  NALCOMIS,  made  it  difficult  for  users  to  establish  a  consistent 
perspective  on  equipment  operation. 

•  The  WWC  display  was  usable,  but  a  tablet  display  might  be  preferable. 

•  Connecting  WWC/VTE  capability  to  other  equipment  might  be  useful. 

•  The  potential  for  VTE  was  seen  as  positive;  NALCOMIS  connectivity  is  essential. 

•  IETM  navigation  could  be  improved. 

Although  the  case  for  using  a  body-worn  computer  for  maintenance  support  was  not  made  in  this 
study,  the  results  must  be  viewed  with  caution,  as  users  had  relatively  little  experience  with  the 
equipment  before  the  termination  of  the  project.  It  was  clear  from  the  user  responses,  however,  that 
mobile  (wireless)  electronic  tools  had  positive  value. 

ENGINEERING  FACTORS 

The  engineering  process  needed  to  realize  that  the  WWC  system  could  be  divided  into  two  parallel 
phases:  technical  and  administrative.  The  technical  tasks  involved  in  the  Tech  Solution  are  detailed  in 
the  following  six  documents: 

1.  Site  Survey  for  Naval  Air  Station  Brunswick  VP-8  Wireless  Maintenance  Network, 

10-1 1  October  2002.  This  document  presents  all  steps  needed  to  ensure  signal  coverage 
throughout  squadron  workspaces  and  optimal  placement  of  wireless  access  points.  Contact 
A.  J.  Ballard  at  207-442-5264  or  Brian  D.  McCue  at  207-447-2803  for  information  about 
acquiring  this  document. 

2.  System  Design  Document  for  Naval  Air  Station  Brunswick  Hangar  5  Wireless  Maintenance 
Network,  23  October  2002.  This  document  contains  the  wireless  system  operating  concept, 
physical  layout,  hardware  and  software  components,  and  installation  descriptions.  Contact 
A.  J.  Ballard  at  207-442-5264  or  Brian  D.  McCue  at  207-447-2803  for  information  about 
acquiring  this  document. 

3.  Hangar  5  Network  Plan  (Figure  B-l).  This  figure  details  the  physical  placement  locations  of  all 
network  components. 

4.  Hangar  5  VPN  Diagram  (Figure  B-2).  This  document  depicts  the  physical  configuration  of  the 
virtual  private  network  for  the  test  site. 
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5.  WWC  System  Specification.  Pen  Systems  provides  the  technical  characteristics  and  physical 
configuration  of  the  WWC  unit  at  the  following  website:  http://www.pensvstems.com/specs.htm 

6.  Getting  Started  With  VirtualBench™.  This  National  Instruments®  document  details  the  core 
software  package  for  supporting  VTE  functions  on  the  WWC.  The  document  is  available  at  the 
following  website:  http://www.ni, com/pdf/manuals/32 151 8e.pdf 

Configuration  of  the  wireless  network  and  VPN  layer  were  completed  with  support  from  Cisco 
Systems,  Inc.,  a  BIW  partner  for  this  project.  Cisco8  components  were  used  extensively  in  the 
network  implementation.  WWC  functionality — particularly  VTE  features — was  realized  with  the 
assistance  of  National  Instruments®  software  and  expertise. 

Obtaining  official  approval  to  operate  the  WWC  system  required  planning  for  safety  and  security 
concerns.  The  key  steps  in  this  effort  are  summarized  in  the  following  four  documents: 

1 .  HERO  Approval  for  Wireless  Local  Area  Network  At  Naval  Air  Station  Brunswick  Maine 
(Appendix  C-l).  This  document  confirms  that  analysis  of  network  operation  presented  no  hazards 
of  electromagnetic  radiation  to  ordnance. 

2.  System  Security  Authorization  Agreement  for  the  Hangar  5  Wireless  Maintenance  Network 
Naval  Air  Station  Brunswick,  26  March  2003.  This  document  contains  a  complete  analysis  of  the 
physical,  organizational,  and  procedural  plans  to  ensure  secure  operations  of  the  wireless 
network,  hosted  on  the  NAS  Brunswick  IT  backbone.  For  information  about  acquiring  this 
document,  contact: 

Commander,  Patrol  &  Reconnaissance  Wing  Five 

5  Jay  Beasley  Circle 

Naval  Air  Station  Brunswick 

Brunswick,  ME  0401 1 

3.  Electromagnetic  Environmental  Effects  (E3)  Review  of  the  Site  Approval  Request  for  Installation 
of  a  Cisco  Wireless  Local  Area  Network  at  Naval  Air  Station  Brunswick  Maine.  This  document 
confirms  the  results  of  an  analysis  that  the  wireless  network  presented  no  electromagnetic 
interference  hazards.  For  information  about  acquiring  this  document  contact: 

Space  and  Naval  Warfare  Systems  Center,  Charleston 
P.  O.  Box  19022 

North  Charleston,  SC  29419-9022 

4.  Interim  Authority  to  Operate  Sensitive  But  Unclassified  Wireless  Information  System  (Appendix 
C-2).  This  document  provides  the  final  step  in  official  approval  to  operate  the  network. 

As  described  earlier,  the  approval  process  was  complicated  by  the  period  in  which  the  Tech 
Solution  project  took  place,  a  time  when  DoD  was  addressing  the  security  issues  of  wireless 
networks  and  attempting  to  generate  guidance  documents.  The  climate  was  characterized  by  extreme 
caution.  Final  approval  provided  authority  to  operate  the  network  at  the  test  site  through  December 
2003.  The  additional  complications  of  early  NMCI  installation  steps  at  NAS  Brunswick  entailed 
additional  (yet,  poorly  defined)  constraints.  The  naval  air  station  now  operates  under  NMCI,  so  new 
operating  authority  must  be  processed  under  additional  rules. 

The  efforts  of  the  Tech  Solution  Team  were  largely  invested  in  obtaining  authorization  to  perform 
the  WWC  tests  (i.e.,  to  operate  a  wireless  network).  This  investment,  however,  has  generated  a 
procedural  and  technical  “road  map”  for  future  efforts  involving  operational  use  of  wireless 
technologies.  At  the  conclusion  of  the  Tech  Solution  project,  all  physical  property  was  assigned  to 
the  host  squadron,  VP-26. 
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DISCUSSION 


Two  benefits  were  gained  from  this  Tech  Solution:  (1)  understanding  of  the  process  for 
implementing  operational  wireless  networks  has  grown  much  more  sophisticated  and  realistic  as  a 
result  of  the  lessons  learned  at  NAS  Brunswick,  and  (2)  a  better  perspective  was  gained  about  how  to 
design  portable  maintenance  support  systems  (i.e.,  alternatives  to  wearable  computers)  as  a  result  of 
user  interviews  and  surveys. 

The  inability  to  gather  performance  data  severely  limited  the  utility  of  the  Tech  Solution  project. 
The  preparation  efforts  necessary  to  stand  up  an  operating  wireless  network  were  difficult,  and  most 
of  the  obstacles  encountered  were  either  unexpected  or  beyond  the  control  of  planners.  As  mentioned 
earlier,  the  baseline  performance  data  were  retained  for  possible  use  in  future  efforts  to  measure 
performance  gains  from  maintenance  support  technologies,  but  this  future  use  is  all  that  can  be 
expected  from  the  performance  measurement  thrust  of  the  project. 

Once  configured,  authorized,  and  activated  in  July  2003,  the  WWC  system  performed  flawlessly 
from  a  hardware  standpoint.  Signal  coverage  of  the  hangar  and  flight  line  were  continuous  through¬ 
out  the  Tech  Solution  test. 

Most  of  the  difficulties  encountered  involved  software  configuration.  Factors  that  limited  the 
quality  of  data  collection  included  the  following  conditions: 

•  Problems  with  NALCOMIS  software  operation  with  wireless  devices;  NALCOMIS  was 
designed  to  work  with  older  UNIX-based  programs.  Differences  between  the  scan  codes  of  a 
physical  keyboard  (which  NALCOMIS  software  was  designed  for)  and  the  wearable  computer 
touch-screen  keyboard  were  very  difficult  to  overcome.  While  this  problem  was  addressed, 
technicians  were  unable  to  perform  their  jobs  with  WWC  and  interact  with  NALCOMIS,  as 
required  by  their  duties.  During  this  period,  collection  of  Maintenance  Task  Results  data  was 
impossible.  BIW  attempted  to  obtain  an  updated  release  of  the  touch-screen  software,  but  project 
funding  was  exhausted. 

•  For  security,  the  project's  infrastructure  was  not  allowed  access  to  a  key  base  router.  Access  to 
this  critical  router  was  required  by  the  wearable  computer  network  for  the  base  Domain  Name 
Server  (DNS)  to  respond  to  the  wireless  network.  Several  weeks  were  lost  attempting  to 
determine  why  the  wireless  network  was  having  DNS-related  issues.  During  this  period,  it  was 
not  possible  to  log  into  the  base  IT  system  to  access  working  directories.  With  no  access  to 
working  directories,  squadron  personnel  could  not  effectively  use  the  wearable  computers. 

•  Time  was  lost  in  gaining  permission  to  modify  the  router  table  with  a  pointer  to  the  wireless 
network  (to  address  the  router  IP  table  issue)  due  to  NAS  Brunswick  security  concerns.  In 
addition,  the  static  IP  route  addition  to  the  single  router  could  not  be  effectively  managed  at  the 
level  of  NAS  Brunswick,  as  all  of  their  router  updates  are  the  secure  global  variety. 

•  The  wireless  network  was  down  for  several  weeks  after  an  NMCI  upgrade  was  applied  to  the 
base  routers.  This  change  was  implemented  without  warning  to  the  Tech  Solution  Team  or  the 
squadron. 

•  Additional  downtime  resulted  from  the  release  of  several  U.S.  Navy  IT  change  notices.  Compli¬ 
ance  with  these  directives  required  the  acquisition  of  software  upgrades  for  the  wireless  network. 
Project  funding  was  exhausted  by  the  time  all  the  new  software  was  obtained. 
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•  A  training/skill  issue  arose  during  the  test,  as  instances  were  found  where  squadron  technicians 
inadvertently  corrupted  some  WWC  settings.  These  problems  had  to  be  painstakingly  tracked 
down  and  fixed,  reducing  time  for  system  operation  and  data  collection. 

•  Command  reluctance  to  use  the  wireless  network  without  NALCOMIS  was  made  clear  to  the 
team.  Nevertheless,  WWC  support  for  NALCOMIS  was  not  part  of  the  original  Tech  Solution 
Statement  of  Work.  Several  weeks  were  lost  in  attempting  to  understand  and  implement 
NALCOMIS  to  satisfy  squadron  requests. 

In  summary,  significant  technical  experience  was  gained  in  the  effort  to  establish  a  wireless 
network  in  an  operational  military  setting.  The  benefit  from  lessons  learned  through  this  effort  should 
not  be  underestimated;  the  knowledge  gained  will  benefit  future  projects  to  introduce  new  technolo¬ 
gies  involving  distributed  personnel  and  a  common  communications/  data  infrastructure. 

Good,  if  limited,  information  was  gained  from  soliciting  user  reactions  to  the  WWC  technology 
concept.  While  complete  faith  should  not  be  placed  in  such  a  small  data  set,  the  trends  in  these  data 
were  enlightening  regarding  the  utility  of  VTE  systems  and  the  best  way  to  present  information  in 
future  efforts  (e.g.,  tablets  or  other  hand-held  devices).  Subsequent  efforts  to  provide  distributed 
maintenance  support  should  include  a  range  of  computing  systems  in  addition  to  (because  the  case  is, 
as  yet,  still  unclear)  wearable  computers. 

TRANSITION 

Representatives  of  Space  and  Naval  Warfare  Systems  Command  (SPAWAR)  and  Naval  Air 
Systems  Command  (NAVAIR)  have  been  contacted  regarding  the  process  for  transitioning  products 
of  this  Tech  Solution  to  program-of-record  status.  While  these  procedures  are  understood,  a  neces¬ 
sary  precursor  to  further  action  is  the  documentation  of  requirements,  beginning  with  endorsements 
from  user  communities.  Given  the  current  lack  of  documented  performance  data  and  the  limited  user 
experience  with  the  system,  serious  pursuit  of  transition  opportunities  without  further  testing  is  not 
recommended. 
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SUMMARY 


This  report  has  presented  evidence  to  justify  the  following  primary  lessons  from  the  Tech  Solution 
project: 

1 .  The  method  for  designing  and  implementing  a  wireless  network  at  an  operational  command  is 
understood  and  validated. 

2.  Electronic  maintenance  manuals  are  useful,  although  no  new  insights  were  gained  through  their 
use  with  WWC  systems. 

3.  Virtual  test  equipment  and  remote  interaction  with  NALCOMIS  are  the  two  capabilities  with  the 
greatest  potential  for  maintenance  support  over  wireless  networks. 

4.  Wireless  support  may  not  require  wearable  computers;  alternate  information  systems  may  be  just 
as  effective. 

Insufficient  data  were  collected  to  provide  a  foundation  for  technology  transition;  additional 
performance  data  are  needed  before  operational  endorsements  or  statements  of  requirements  can  be 
solicited. 
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APPENDIX  A 
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APPENDIX  A-1 

WWC/VTE  SYSTEM  EVALUATION 


Notes: 

1 .  Large  X  =  average  of  original  survey  respondents 

2.  Small  X  =  average  of  final  survey  respondents 

3.  Specific  averages  (original  and  final)  are  found  below  the  scale  for  each  item 

4.  Comments  from  technicians  are  from  final  survey  only;  no  comments  were  generated 
during  original  survey. 

5.  Most  comments  came  from  a  single  individual  (AT2) 


GENERAL  CHARACTERISTICS 

1.  This  equipment  is 

Too  heavy  Light  enough 


for  regular  individual  use 

Original  rating  average  =  2.6;  final  rating  average  =  2.7 
Comments: 

Light  enough,  but  too  bulky  for  practical  use 


2.  This  equipment  is 

Too  bulky 


Compact  enough 


X - | - X 


for  regular  individual  use 

Original  rating  average  =  2.3;  final  rating  average  =  1.7 
Comments: 

Hampers  more  than  it  helps 


3.  The  WWC  system  is  at  least  as  reliable  as  any  other  maintenance  equipment  I  use 

Strongly  agree  No  opinion  Strongly  disagree 


Original  rating  average  =  2.0;  final  rating  average  =  2.4 
Comments: 

Functionality  is  always  questionable;  Software  is  big  issue;  LAN  connection  is  spotty  at  best 
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4.  The  VTE  system  is  at  least  as  reliable  as  any  other  maintenance  equipment  I  use 


Strongly  agree  No  opinion  Strongly  disagree 


Original  rating  average  =  2.0;  final  rating  average  =  2.3 
Comments: 

VTE  is  the  best  concept  of  the  system;  saves  us  time  and  saves  the  Navy  money 


5.  All  parts  of  this  equipment  (e.g.,  controls,  leads,  etc.)  are  easy  to  reach  when  I  need 
them 

Strongly  agree  No  opinion  Strongly  disagree 


Original  rating  average  =  2.0;  final  rating  average  =  1 .8 
Comments: 

Layout  is  decent.  Belt  and  shoulder  harness  get  in  the  way  of  maintenance.  Meters  are  nice  if 
user  is  properly  trained. 

Didn’t  have  a  complete  vest  with  oscilloscope  card  or  anything. 


6.  I  think  that  all  parts  of  this  equipment  are  easy  to  stow  on  the  vest  when  I  don’t  need 
them 

Strongly  agree  No  opinion  Strongly  disagree 


Original  rating  average  =  2.0;  final  rating  average  =  1 .8 
Comments: 

Plan  is  good;  actual  use  is  cumbersome. 


7.  I  think  that  this  equipment  is  easy  to  store  in  the  work  center  between  uses 

Strongly  agree  No  opinion  Strongly  disagree 

| - | - X  -X  | . | . | 

Original  rating  average  =  1.9;  final  rating  average  =  1.7 
Comments: 

Weren’t  allowed  to  store  them  in  the  work  center. 
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8.  I  think  that  this  equipment  is  rugged  enough  to  function  under  all  operational 
maintenance  conditions 

Strongly  agree  No  opinion  Strongly  disagree 


Original  rating  average  =  2.0;  final  rating  average  =  2.3 
Comments: 

9.  I  think  that  the  battery  life  is  sufficient  for  completing  most  jobs 

Strongly  agree  No  opinion  Strongly  disagree 


Original  rating  average  =  2.0;  final  rating  average  =  1 .8 
Comments: 


10.  I  think  that  the  battery  is  easy  to  replace  when  needed 

Strongly  agree  No  opinion  Strongly  disagree 


Original  rating  average  =  2.0;  final  rating  average  =  1.5 
Comments: 


DISPLAY  CHARACTERISTICS 

1.  I  can  see  the  display  surface  under  all  conditions 

Strongly  agree  No  opinion  Strongly  disagree 


|X-X 


Original  rating  average  =  2.1;  final  rating  average  =  2.3 
Comments: 


2.  I  think  that  the  display  can  be  used  during  work  without  unnecessarily  tying  up  my 
hands 

Strongly  agree  No  opinion  Strongly  disagree 


Original  rating  average  =  2.0;  final  rating  average  =  2.1 
Comments: 

Tablet  display  would  be  better  (that  we  could  set  down).  We  have  no  need  to  actually  wear 
the  computer. 
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3.  I  can  read  the  text  information  on  the  display  under  all  conditions 


Strongly  agree  No  opinion  Strongly  disagree 

| - | - |  X-X - 1 - | 

Original  rating  average  =  2.1;  final  rating  average  =  2.3 
Comments: 


4.  I  can  read  the  graphical  information  (i.e.,  symbols,  figures)  on  the  display  under  all 
conditions 

Strongly  agree  No  opinion  Strongly  disagree 

| - | - |  X-X - 1 - | 

Original  rating  average  =  2.1;  final  rating  average  =  2.3 
Comments: 


5.  I  think  that  color  is  useful  for  reading  the  display 

Strongly  agree  No  opinion  Strongly  disagree 

| - |  ...„x - X  | - 1 - | 

Original  rating  average  =  1.9;  final  rating  average  =  1.3 
Comments:  1.9  1.3 


6.  I  would  suggest  the  following  changes  to  improve  the  physical  display: 

No  comments  to  this  item 


7.  I  would  suggest  the  following  changes  to  improve  the  information  on  the  display: 

Better  software  functionality 


8.  I  think  that  the  system  controls  provide  enough  range  to  adjust  the  display  under  all 
conditions 

Strongly  agree  No  opinion  Strongly  disagree 


-X-X 


Original  rating  average  =  2.0;  final  rating  average  =  1 .9 
Comments:  2.0  1.9 


9.  I  would  suggest  the  following  changes  to  the  display  controls  (e.g.,  thumbwheels, 
pushbuttons,  knobs,  different  ranges  of  adjustment,  functions,  etc.): 

No  responses  to  this  item 
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SYSTEM  CHARACTERISTICS 


1.  I  think  that  the  system  should  be  able  to  utilize  aircraft  displays  by  tying  into  aircraft 
access  points 

Strongly  agree  No  opinion  Strongly  disagree 

| - | - X - X  | - 1 . | 

Original  rating  average  =  1.9;  final  rating  average  =  1.6 
Comments: 

Doesn’t  seem  necessary  for  our  purposes 


2.  I  think  the  system  should  be  able  to  connect  and  use  other  devices  (e.g.,  conventional 
test  equipment,  other  data  terminals,  etc.) 

Strongly  agree  No  opinion  Strongly  disagree 

|  |  xX  |  |  | 

Original  rating  average  =  1.7;  final  rating  average  =  1.6 
Comments: 

This  is  one  of  the  best  functions  of  this  system.  Integration  of  equipment  we  use  could  save 
us  time. 


3.  Self-calibration  is  an  important  and  useful  feature  of  the  VTE  susbsystem 

Strongly  agree  No  opinion  Strongly  disagree 

Original  rating  average  =  1.8;  final  rating  average  =  1.5 
Comments: 

Self  calibration  makes  everything  easier. 


4.  I  would  make  the  following  suggestions  to  improve  the  features  (e.g.,  controls  and 
peripheral  devices)  of  the  WWC  system 

Just  give  us  tablets.  We  have  no  need  to  wear  the  computers;  not  wearing  them  would 
increase  their  use  exponentially. 

NALCOMIS 


5.  I  would  make  the  following  suggestions  to  improve  the  features  (e.g.,  controls  and 
peripheral  devices)  of  the  VTE  subsystem 

VTE  is  very  nice  and  effective 
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ELECTRONIC  MAINTENANCE  MANUALS 


Navigation 

1.  The  controls  are  effective  and  appropriate  for  finding  the  manual  pages  I  need 

Strongly  agree  No  opinion  Strongly  disagree 


Original  rating  average  =  2.0;  final  rating  average  =  2.1 
Comments: 


2.  I  can  locate  the  manual  pages  I  need  quickly 

Strongly  agree  No  opinion  Strongly  disagree 


Original  rating  average  =  2.0;  final  rating  average  =  2.1 
Comments: 


3.  I  always  know  where  I  am  in  the  manual 

Strongly  agree  No  opinion  Strongly  disagree 


Original  rating  average  =  2.0;  final  rating  average  =  1 .8 
Comments: 


4.  The  best  feature  to  help  me  navigate  through  the  manuals  is 

Windows  /  Adobe  reader 
Index 


5.  The  worst  navigation  feature  is 

Getting  used  to  the  electronic  keyboard 


6.  I  suggest  the  following  changes  to  better  help  me  navigate  through  the  manuals  (e.g., 
controls,  label  information,  etc.) 

No  responses  to  this  item 
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Display 


7.  I  like  the  way  that  information  is  organized  and  presented  on  the  display;  pages  are 
divided  up  in  a  logical  manner 

Strongly  agree  No  opinion  Strongly  disagree 


Original  rating  average  =  2.0;  final  rating  average  =  1 .8 
Comments: 


8.  I  can  read  the  graphical  information  (e.g.,  diagrams  and  charts)  without  difficulty  while 
doing  my  job 

Strongly  agree  No  opinion  Strongly  disagree 


Original  rating  average  =  2.0;  final  rating  average  =  1 .8 
Comments: 


9.  The  best  feature  of  the  manual  display  is 

Size  and  visibility 


10.  The  worst  feature  of  the  manual  display  is 

It’s  attached  to  the  user’s  body  (totally  unnecessary) 


11.1  suggest  the  following  changes  to  improve  the  display  of  maintenance  manuals 

More  accessibility  to  updated  pubs 


Content 

12.  The  manuals  I  use  have  all  the  material  I  need  to  support  my  job  performance 

Strongly  agree  No  opinion  Strongly  disagree 


Original  rating  average  =  2.0;  final  rating  average  =  1 .9 
Comments: 


13.  I’d  also  like  to  see  the  following  support  documents  available  on  this  system  (e.g., 
NATOPS,  MIMS,  etc) 

Comprehensive  pubs  are  vital. 

ALL 

CSMM’s 
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14.  I  suggest  the  following  changes  to  the  content  of  these  manuals  to  better  help  me  do 
my  job  (e.g.,  functionality,  content,  web  browsing,  etc.) 

Different  systems 

Data  Access  Strategy 

15.  Manuals  and  other  documents  should  be 

2  loaded  on  the  wearable  computer 

2  accessed  from  a  central  database,  to  be  viewed  on  the  wearable  computer 
2  a  combination  of  both,  depending  on  the  job 
Note:  only  six  people  responded  to  this  item 
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APPENDIX  A-2 

MAINTENANCE  TASK  Results 

1 .  Where  was  this  job  primarily  performed? 

_ flight  line  _ hangar  _ work  center 

2.  When  was  it  performed? 

_ day  _ night  _ dawn /dusk 

3.  What  were  you  asked  to  do? 

4.  This  task  was  _ preventive  maintenance  _ unscheduled  maintenance 

5.  What  did  you  need  to  accomplish  to  complete  this  job? 

Test  equipment _ 

Manuals _ 

Parts _ 

Other _ 

6.  What  was  the  outcome  of  this  job? 

_ Complete  _ AWM  _ AWP  _ Other  (  please  specify: _ ) 

7.  How  much  time  did  you  spend  on  this  job  (you  can  record  start/stop  times  if  you  like)? 

8.  Did  you  need  to  work  with  other  technicians  to  complete  this  job?  If  so,  who  (i.e., 
rating  and  rate)? 

9.  Did  you  need  the  help  of  your  supervisor  to  complete  this  job? 

10.  Did  you  need  to  return  to  your  work  center  while  performing  this  job?  If  so,  how  many 
times? 

Why? 
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APPENDIX  A-3 
WWC/VTE  RESULTS 


_ The  system  failed  during  this  job 

Nature  of  failure: 


1.  _ I  needed  to  replace  the  battery  before  starting  this  job 

2.  _ I  needed  to  replace  the  battery  during  this  job 

3.  Please  mark  your  best  judgment  regarding  the  usefulness  of  the  WWC/VTE  system  for 
this  job  - 

Ineffective  /  the  same  as  Productive  / 

wasteful  working  manually  helpful 


4.  The  most  effective/productive  feature  of  this  system  (for  this  job)  was  - 


5.  The  least  effective/productive  feature  (for  this  job)  was  - 


6.  Other  comments  about  the  performance  or  usability  of  the  WWC/VTE  system  (for  this 
job)  - 


7.  I  would  suggest  the  following  changes  to  improve  the  WWC/VTE  - 
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HAZ* 

Hangar  5  South  Cat5  drop  diagram 

Drops  placed  in  middle  of  wall  or  follow  old  wiring  placement,  whichever  is  easier.  Both  Red  &  Blue  dots  =  Cat5,  but 
designate  separate  LAN’s  eventually.  Diagrams  are  not  all  to  scale,  and  should  be  used  for  placement  data  only. 

tin  LAN  •  (Red)=65,  Nalcomis  LAN  •  (Blue)=17,  Router  (R)=2,  Switch  (S)=2,  Hub  (H)=7. 

_ 0  Yellow  X  =  requested  extra  drops  if  material  is  left  over  after  primary  job,  total  14 _ 
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Hangar  5  North  Cat5  drop  diagram 

Drops  placed  in  middle  of  wall  or  follow  old  wiring  placement,  whichever  is  easier.  Both  Red  &  Blue  dots  =  Cat5,  but  designate 
separate  LAN’s  eventually.  Diagrams  are  not  all  to  scale,  and  should  be  used  for  placement  data  only. 

Admin  LAN  •  (Red)=66,  Nalcomis  LAN  •  (Blue)=17,  Hub  (H)=l. 

0  Yellow  X  =  requested  extra  drops  if  material  is  left  over  after  primary  job,  total  13 


Figure  B-1 .  Hangar  5  Cat5  drop  diagram  (Top:  South:  Bottom:  North) 
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CPRW-5 

Virtual 

Private 

Network 

(205 . 69 . xxx . xx) 


*  \ 


>  k 


EO  =  205.69.225.1 


E0  =  205.69.226.1 


EO  =  205.69.227.1 


Figure  B-2.  Hangar  5  Cat5  drop  diagram  (Top:  South:  Bottom:  North) 
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APPENDIX  C-1 


DEPARTMENT  OF  THE  NAVY 

NAVAL  ORDNANCE  SAFETY  &  SECURITY  ACTIVITY 
FARRAGUT  HALL  BLDG  D-323 
23  STRAUSS  AVENUE 
INDIAN  HEAD  MD  206404555 

8  0  2  0 

Ser  N7162/1422 

December  2,  2002 

B.  McCue 

Bath  Iron  Works 

700  Washington  Street 

Bath,  ME  04530-2556 

Dear  B .  McCue : 


SUBJECT :  HERO  APPROVAL  FOR  WIRELESS  LOCAL  AREA  NETWORK  AT 

NAVAL  AIR  STATION  BRUNSWICK  MAINE 

In  accordance  with  NAVSEA  OP  3565/NAVAIR  16-1-529,  Volume  2, 
Eleventh  Revision,  a  Hazards  of  Electromagnetic  Radiation  to 
Ordnance  (HERO)  analysis  was  conducted  to  determine  the  impact  of 

installing  and  utilizing  a  Wireless  Local  Area  Network  (WLAN)  in 

Hangar  5  at  Naval  Air  Station  Brunswick,  Maine.  The  purpose  of 

the  analysis  was  to  determine  if  this  installation  has  any 
'adverse  impact  upon  local  ordnance  items.  New  installations, 

such  as  WLAN '  s  with  access  point  transceivers  and  client  adapter 
network  cards  can  affect  the  electromagnetic  environment  and  have 

adverse  impact  on  ordnance  items  within  the  area. 

The  analysis  detailed  in  enclosure  (1)  indicates  HERO  issues  will 

not  be  a  concern  for  the  antenna  of  the  fixed  access  points  ; 

however,  HERO  Emission  (EMCON)  control  will  be  required  for  the 
client  adapters .  The  EMCON  requirement  will  be  to  maintain  a 

safe  separation  distances  when  HERO  UNSAFE  or  HERO  SUSCEPTIBLE 
ORDNANCE  is  within  the  area .  If  the  distance  cannot  be 

maintained  then  the  offending  client  adapter  must  be  silenced. 

The  required  safe  separation  distances  are  listed  in  enclosure 
(1) 

hero  approval  is  granted  for  this  installation  provided  the  safe 
separation  distances  specified  in  enclosure  (1)  are  maintained. 


Sincerely 


Director,  Explosives  Safety  and 
Security 

By  direction  of  the 
Commanding  Officer 


//Soo9^i/ 
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8020 

Ser  N7162/1422 

December  2 ,  2002 

Enclosure:  1.  NAVSURFWARCENDIV  Dahlgren  ltr  8020  Ser  J52/1288 

Of  21  Nov  02 

copy  to : 

NAS  Brunswick  (Safety  Officer,  GEMD) 

NOSSA  ESSOLANT  (N7L) 

NAVSURFWARCENDIV  Dahlgren  (J52 /Denham) 

EG&G  Dahlgren  (E.  Kuhn,  D.  Willis) 
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DEPARTMENT  OF  THE  NAVY 

NAVAL  SURFACE  WARFARE  CENTER 
DAHLGREN  DIVISION 
17320  DAHLGREN  ROAD 
DAHLGREN,  VIRGINIA  22448-5100 

IN  REPLY  REFER  TO 

8020 

Ser  J52/1288 

MOV  2  1  2002 

From:  Commander,  Dahlgren  Division,  Naval  Surface  Warfare  Center 

To:  Commanding  Officer,  Naval  Ordnance  Safety  and  Security 

Activity 

(N716/C.  Wakefield) 

Farragut  Hall,  Bldg.  D323 

23  Strauss  Avenue 

Indian  Head,  MD  20640-5555 

Subj  :  HAZARDS  OF  ELECTROMAGNETIC  RADIATION  TO  ORDNANCE  (HERO) 

ANALYSIS  FOR  THE  INSTALLATION  OF  A  WIRELESS  LOCAL  AREA 
NETWORK  (WLAN)  IN  HANGAR  5  AT  NAVAL  AIR  STATION  (NAS) 
BRUNSWICK,  MAINE 


Ref : 

(a) 

NAVSEA 

OP  3565/NAVAIR  16-1-529,  Volume 

2,  Eleventh 

Revision 

of  1  May  02 

(b) 

E-Mail 

Bath  Iron  Works,  B.  McCue/EG&G 

Technical 

Services, 

Inc.,  R.  Mersiowsky  of  6  Nov 

02 

(c) 

NAVORDCEN  ltr  8020  Ser  N7132/215  of  28 

Mar  97 

Enel : 

(1) 

System 

Specifications  and  Hazards  of 

Electromagnetic 

Radiation 

to  Ordnance  Safe  Separation 

Distances 

(2) 

Hangar 

5  Wireless  Local  Area  Network 

System  Antenna 

Location 

Diagram 

(3) 

Recommended  Distribution  List 

1  The  subject  analysis  was  conducted  in  accordance  with 

reference  (a)  as  requested  by  reference  (b)  concerning  the  HERO 

impact  of  installing  and  operating  a  WLAN  in  Hangar  5  at  NAS 
Brunswick,  Maine.  This  request  and  the  system  specifications 

were  provided  on  6  November  2002.  Enclosure  (1)  provides  the 

system  specifications  and  HERO  safe  separation  distances. 

Enclosure  (2)  is  a  diagram  showing  the  locations  of  the  fixed 

access  points  within  Hangar  5.  Enclosure  (3)  contains  a 

recommended  distribution  list. 

2  .  The  WLAN  system  will  be  installed  at  multiple  locations 

within  Hangar  5.  The  system  includes  fixed  access  point 
transceivers  and  client  adapter  network  interface  cards 

installed  in  a  variety  of  computers  throughout  the  facility. 
Enclosure  (1)  provides  the  system  specifications  and  HERO  safe 


r  \ ) 
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Sub]:  HAZARDS  OF  ELECTROMAGNETIC  RADIATION  TO  ORDNANCE  (HERO) 

ANALYSIS  FOR  THE  INSTALLATION  OF  A  WIRELESS  LOCAL  AREA 
NESWCKK  (WLAN)  IN  HANGAR  5  AT  NAVAL  AIR  STATION  (NAS) 
BRUNSWICK,  MAINE 

separation  distances.  Enclosure  (2)  is  a  diagram  showing  the 
locations  of  the  fixed  access  points  within  Hangar  5 . 

3  .  Calculated  electromagnetic  environments  indicate  that  the 
antenna  placement  of  the  fixed  access  points  listed  in 
enclosure  (2)  is  such  that  the  distance  and/or  radiation  angle 
with  respect  to  ordnance  locations  preclude  the  need  for  HERO 
emission  control  (EMCON)  . 

4.  HERO  EMCON  will  be  necessary  for  the  client  adapters  listed 
in  enclosure  (1)  whenever  ordnance  operations  occurwithin  the 
HERO  safe  separation  distances  provided.  EMCON  will  consist  of 

silencing  the  offending  client  adapter, 

5  It  is  recommended  that  enclosure  (1)  be  incorporated  into 
Appendix  A  of  reference  (c),  the  current  HERO  assessment  report 
for  this  station. 

6  ,  Hiis  project  is  recommended  for  HERO  approval  provided  there 
is  adherence  to  the  HERO  safe  separation  distances  specified  in 
enclosure  (1)  for  the  client  adapter. 

7  If  there  are  any  questions  or  comments,  please  contact 

Charles  Denham,  J52,  at  commercial  (540)  653-3444  or 

DSN  249-3444,  or  via  electronic  mail  at  denhamcc@nswc.navy.mil. 


VIRGINIA  S.  HUDSON 

By  direction 
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SYSTEM  SPECIFICATIONS  AND 

HAZARDS  OF  ELECTROMAGNETIC  RADIATION  TO  ORDNANCE 
SAFE  SEPARATION  DISTANCES 


Enclosure  (1) 


C-6 


SYSTEM  SPECIFICATIONS  AND  HERO  SAFE  SEPARATION  DISTANCES 


Separation  Distances 


Antenna 

Number 

Antenna 

Nomenclature 

Antenna  Type 

Antenna 

Gain 

(dBi) 

Transmitter 
Transmitter  Max.  Avg. 

Frequency  (MHz)  Power  (watts) 

Transmitter  Type 

HERO  UNSAFE 
ORDNANCE 
(feet/meters) 

HERO 

SUSCEPTIBLE 

ORDNANCE 

(feet/meters) 

A1 

AIR-ANT1949 

Yagi 

13.5 

2400  - 

2490 

0.1 

Cisco  Aironet  350  A P 

10/3 

5/1.5 

A2 

AIR-ANT2506 

Omni 

5.2 

2400  - 

2490 

0.1 

Cisco  Aironet  350  AP 

10/3 

5/1.5 

A3 

AIR-ANT1949 

Yagi 

13.5 

2400  - 

2490 

0.1 

Cisco  Aironet  350  AP 

10/3 

5/1.5 

A4 

AIR-ANT3549 

Patch 

8.5 

2400  - 

2490 

0.1 

Cisco  Aironet  350  AP 

10/3 

5/1.5 

A5 

AIR-ANT3549 

Patch 

8.5 

2400  - 

2490 

0.1 

Cisco  Aironet  350  AP 

10/3 

5/1.5 

Mobile 

PCs 

AIR-PCM35X 

Diversity  Dipole 

1 

2400  - 

2490 

0.1 

Cisco  Aironet  350 
Client  Adapters 

1/0.3 

1/0.3 

C-7 


8020 

Ser J52/1288 


HANGAR  5  WIRELESS  LOCAL  AREA  NETWORK  SYSTEM 
ANTENNA  LOCATION  DIAGRAM 


Enclosure  (2) 
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HANGAR  5  WLAN  SYSTEM  ANTENNA  LOCATION  DIAGRAM 
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RECOMMENDED  DISTRIBUTION  LIST 


Enclosure  (3) 


C-10 


RECOMMENDED  DISTRIBUTION  LIST  FOR  HERO  ANALYSIS 
FOR  THE  INSTALLATION  OF  A  WLAN  IN  HANGAR  5  AT  NAS  BRUNSWICK,  ME 


Bath  Iron  Works 
(B.  McCue) 

700  Washington  Street 
Bath,  ME  04530-2556 

Commanding  Officer 
Naval  Air  Station 
(Safety) 

(GEMD) 

1251  Orion  Street 
Brunswick,  ME  04011-5009 

Commander 

Naval  Ordnance  Safety  and  Security  Activity 
Explosives  Safety  Support  Office  Atlantic 
(N7A) 

1282  Gilbert  Street 
Norfolk,  VA  23511-2496 

EG&G  Technical  Services,  Inc. 

(E.  Kuhn) 

(HERO  Data  Base) 

16156  Dahlgren  Road,  P.O.  Box  552 
Dahlgren,  VA  22448-0552 
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APPENDIX  C-2 


2000 
Ser  Nl/ 

From:  Commander,  Patrol  and  Reconnaissance  Wing  FIVE 

To:  Command  Information  Systems  Security  Manager  (N62A) 

Sub j :  INTERIM  AUTHORITY  TO  OPERATE  SENSITIVE  BUT  UNCLASSIFIED 
WIRELESS  INFORMATION  SYSTEM 

Ref:  (a)  OPNAVINST  5239. IB 

(b)  SECNAVINST  5239.3 

(c)  CINCLANTFLTINST  5239. 1A 

Enel:  (1)  System  Security  Authorization  Agreement  (SSAA) 

(2)  Current  HERO  Certification 

(3)  SPAWAR  E-mail  on  HERF  and  HERP  Certifications 

1.  Per  references  (a)  through  (c) ,  the  Sensitive  but  Unclassified 
(SBU)  Wireless  Telemaintenance  Information  System  within  CPRW-5's 
claimancy  is  granted  an  Interim  Authority  to  Operate  (IATO)  for  a 
period  of  one  year  from  the  date  of  this  document,  or  until  NMCI 
assumes  the  project,  whichever  comes  first.  During  this  interim 
period,  a  full  accreditation  determination  should  be  made  within 
CINCLANTFLT  or  by  designated  sponsoring  activity  based  upon  the 
completed  DITSCAP  requirements  of  enclosure  (1),  as  well  as 
operational  test  data  as  evaluated  and  reported  by  the  sponsoring 
ONR  lab,  SPAWAR  Systems  Center  -  San  Diego. 

2.  Enclosure  (1)  is  the  completed  SSAA,  enclosure  (2)  reflects 
the  Wireless  Systems'  HERO  certification,  and  enclosure  (3) 
reflects  SPAWAR' s  verbal  approval  of  the  Wireless  Systems'  HERP  and 
HERF  certifications,  documentation  forthcoming. 

3.  This  IATO  covers  the  following  system: 

Hangar  5  North,  Wireless  Telemaintenance  Network,  which 
consists  of: 
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a.  Wearable  or  portable  computers 

b.  Wireless  2.4Ghz  Client  NIC  Cards 

c.  2.4Ghz  Access  Points 

d.  Cisco  PIX  Firewall 

e.  Cisco  VPN  concentrator 

f.  Cisco  Catalyst  LAN  Switches 

g.  Cisco  MCS  Call  Manager 

h.  Cisco  Hard  IP  Phones 

Sub j :  INTERIM  AUTHORITY  TO  OPERATE  SENSITIVE  BUT  UNCLASSIFIED 
WIRELESS  INFORMATION  SYSTEM 

i.  Cisco  Software  Utilities 

j .  National  Instruments  Data  Acquisition 

k.  Test  Cisco  2.4Ghz 

l .  Antennas 

m.  Miscellaneous  portable  computer  attachments  (cameras 
and  headsets  and  miscellaneous  software) 

n.  GOTS  Windows  2000  Pro  software  load. 


J.  C.  GRUNEWALD 

Copy  to:  (w/o  end) 

VP  8  (CO/ISO/ISSM) 

VP  10  (CO/ISO/ISSM) 

VP  26  (CO/ISO/ISSM) 

VPU  1  (ISO/ISSM) 

Additional  endorsements  to  this  IATO:  (In  no  particular  order) 
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